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ABSTRACT 



Because of the Job Training Partnership Act (JTPA) 



program's strong emphasis on performance, there is a danger that 
local JTPA programs are neglecting the program's goal of adequately 
serving those with the greatest problems in obtaining and sustaining 
employment. A number of JTPA officials at the federal, state, and 
local levels were interviewed to determine ways in which the JTPA 
performance standards system can be used to encourage the provision 
of services to hard-to-employ individuals while still maintaining an 
emphasis on performance* Two types of policies were determined to be 
promising: (1) reducing disincentives to serve hard-to-serve persons 
and (2) providing incentives to encourage increased service to 
hard-to-serve populations. Policies to reduce disincentives include 
adding factors to the models for deficiencies and barriers, 
additional target groups, and adjustments for activities; changing 
the reward structure for exceeding standards; and providing 
performance standards exemptions for hard-to-serve individuals. 
Policies to provide additional incentives include allocating 
six-percent funds, adding standards and giving extra credit for the 
hard-to-serve, and providing additional technical assistance. 
(Reports on the Illinois Pilot Performance Standards Adjustment 
Models and extra credit for participants with barriers to employment 
are appended.) (MN) 
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Executive Summary 



The Job Training Partnership Act (JTPA) is the nation 1 s major Federal 
employment and training program. Title II -A of JTPA authorizes programs for 
economically disadvantaged youth and adults. JTPA emphasizes program 
performance and the provision of services to individuals with major labor 
market problems. Because performance is measured by outcomes such as 
entered employment rates, wages at placement, and the average cost per 
participant, there is a danger that the strong emphasis on performance can 
lead to neglect of the goal of adequately serving those with the greatest 
problems in obtaining and sustaining employment. This report focuses on 
ways the JTPA performance standards system (i.e., botn performance standards 
and performance incentives that reward good performance) can be used to 
encourage services to such individuals while still maintaining an emphasis 
on performance. The report is based on a series of interviews with 
knowledgeable JTPA officials at the Federal, State, and local levels. The 
goal of the report is to stimulate consideration of the issues and suggest a 
number of approaches that can be used to encourage increased services to 
individuals who may be considered hard- to -serve . 

The question of whether the program is adequately serving hard-to- 
serve individuals is complicated by several factors. First, the statute 
does not define the term hard- to -serve . Second, there may be problems in 
measuring the characteristics associated with hard-to-serve status. As a 
first step, we propose the following definitions to help put the issues in 
perspective: 

Hard-to-serve. This term is used in the report to identify individuals with 
labor market deficiencies or barriers to employment. The most common 
deficiencies are thought to be a lack of basic skills, particularly reading 
skills, and a lack of work experience. 

Difficult- to-place. This term, which is not used in the statute, describes 
groups for whom evidence has shown that SDAs are likely to obtain below 
average placement rates. Examples include women, welfare recipients, and 
members of minority groups. The lower placement rates for these groups may 
result becau' a disproportionate share are hard-to-serve or because of 
discrimination in the labor market. 

Most in need. Individuals can be considered most in need of services based 
either on their preprogram level of family income or their responsibilities 
for supporting a family. The statute does not define this term, and States 
or service delivery areas (SDAs) may develop their own definition. 

Note that we have defined hard- to -serve status on the basis of individual 
characteristics that are more difficult to define and measure than the group 
characteristics used to define difficult- to-place status. 

The performance standards system in JTPA is one of the key features of 
JTPA that influences who is served. Performance standards are used to 
measure the success of individual SDAs, so SDAs interested in demonstrating 
their success rely on their measured performance. In addition, high and low 
performance are recognized by awards and sanctions, respectively. Finally, 
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the Department of Labor has stressed the importance of performance through 
its regulations and issuances. Thus, the structure of the performance 
standards system plays an important role in determining who is served. 

Most States have adopted performance standards systems based on the 
optional adjustment models developed by the Department of Labor. These 
models have been developed to hold SDAs harmless for serving individuals 
with characteristics associated with higher costs and lower placement rates. 
While the models help to avoid disincentives for serving the groups that are 
included (e.g., women, minority groups, dropouts, handicapped, and welfare 
recipients), the current models do not include measures of labor market 
deficiencies or barriers to employment that characterize hard- to -serve 
individuals. In addition, because the models are constructed to hold SDAs 
harmless for serving individuals in these groups, they do not provide net 
incentives to serve the hard- to -serve. The report suggests two types of 
policies that can be made: (1) reducing disincentives to serve the hard-to- 
serve, and (2) providing incentives to encourage increased service to this 
population. Policies to reduce disincentives to serve the hard-to-serve 
include : 

Adding factors to the models for deficiencies and barriers . Virtually 
everyone consulted in the course of this study stated that demographic 
factors do not adequately measure how hard-to-serve an individual is. There 
was also a general consensus that the key deficiencies are a lack of reading 
and math skills and little or no work experience. The Department of Labor 
has proposed to collect information on basic skills deficiencies on the JTPA 
Annual Status Report (JASR) for the purpose of adding these factors to the 
adjustment models. States may wish to consider adding their own adjustments 
until national models are developed. Model adjustments can also be made for 
barriers such as a lack of day care and adequate transportation. 

Adding additional target groups to the adjustment models . Some States have 
encouraged services to additional target groups by adding factors to the 
performance standards adjustment models. Illinois, for example, added 
offenders, teenage parents, rural clients, and displaced homemakers. 
However, it should be kept in mind that many people believe that identifying 
the hard-to-serve on the basis of demographic characteristics is not as 
useful as measuring the underlying deficiencies and barriers. 

Adding adjustments for activities . Hard-to-serve participants are often 
placed in special remedial programs to improve their basic skills. Until 
factors measuring such deficiencies are included in the models, States may 
wish to add factors to the models measuring the proportion of participants 
enrolled in such special programs. We are not considering here the more 
general discussion about whether activity factors should be included in the 
models, but rather only factors for programs targeted on the hard-to-serve. 

Changing the reward structure for exceeding standards . States have latitude 
in the manner in which they reward SDAs for the extent to which they exceed 
the standards. States that wish to encourage services to the hard-to-serve 
may wish to place less emphasis in their awaiis system on the extent to 
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which standards are exceeded. Some States limit the extra award that an SDA 
can receive once the standards are exceeded by a specified amount. 

The use of Governor's adjustments . Governors are permitted to lower the 
minimum level of acceptable performance for reasons including encouraging 
service to hard-to-serve individuals. By reducing the minimum level of 
acceptable performance, SDAs are less likely to worry about failing to meet 
the standards when they serve hard-to-serve individuals. 

Providing performance standards exemptions for the hard- to -serve . States 
may wish to encourage SDAs to undertake projects with hard-to-serve 
individuals by exempting particular projects from performance standards as 
part of a "Governor's adjustment." This approach should be used with 
caution because it removes the incentives provided by the performance 
standards system to focus on outcomes. For untried, innovative programs 
dealing with especially hard-to-serve clients, it may be necessary to 
provide strong inducements for SDAs to take risks. 

Regional or State modeling . Regions or States can develop their own 
adjustment models to improve targeting on the hard- to -serve as well as for 
other purposes. State or regional models such as the ones being developed 
in Region V can be used to assure that model adjustments reflect conditions 
specific to the region rather than national averages. Also, by tailoring 
the data collected, the model can incorporate measures of deficiencies or 
barriers judged to be important. 

Policies to provide additional incentives to serve the hard-to-serve 
include: 

Allocation of six-percent funds . States can use six percent of their 
allocation for performance awards, providing technical assistance, and for 
incentive payments for serving hard-to-serve individuals. States that use a 
greater share of these funds to reward performance and less to encourage 
services to the hard-to-serve provide a relatively stronger incenti" e for 
emphasizing performance over services to the hard-to-serve if other things 
are equal. To emphasize the use of six-percent funds for the hard-to-serve, 
States should clearly specify what awards will be made for various levels of 
services to the target groups. 

Adding standards for the hard- to -serve . A clear way for a State to stress 
interest in the hard-to-serve is to add one or more standards. These 
standards can relate to service levels and/or outcomes. At least two States 
have added standards that require SDAs to meet service levels and to achieve 
adequate entered employment rates for the target groups to qualify for 
certain financial awards. 

Giving extra credit for the hard- to -serve . An interesting approach that to 
our knowledge has not yet been tried is for States to give extra credit in 
computing performance scores for placing individuals designated as being 
hard-to-serve. For example, in computing an SDA f s entered employment rate 
the SDA could receive credit for 1.5 or 2 placements for each hard-to-serve 
participant that terminates with a job. A major advantage of this approach 
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is that SDAs would have strong incentives to place hard-to-serve individuals 
as well as to serve them. The major problem with this approach is that it 
involves keeping two sets of records-- one set to report who was served and 
another for computing performance scores. 

Modifying factor weights for hard-to-serve groups . States may wish to 
change the weights to give an extra incentive to serve individuals with 
particular characteristics. If, for example, a State made the weight for 
welfare recipients in the entered employment rate model more negative, SDAs 
might be willing to serve more welfare recipients because they would receive 
a bigger break in their expected performance. 

Providing technical assistance for encouraging services to the hard-to- 
serve . One use of a State's six-percent funds is to provide technical 
assistance to SDAs. Technical assistance can be used to explain the State's 
priorities for the hard- to -serve and to make sure that SDAs understand the 
incentives the State has provided. Such assistance will be especially 
helpful when a State has adopted a policy that has not been widely used in 
the past. States are permitted to provide "preventative" technical 
assistance for program improvement, so a State interested in encouraging 
services for the hard-to-serve need not wait until a problem develops. 

The approaches described above are not mutually exclusive. States 
could select a number of the suggestions to meet their policy objectives. 
The Department of Labor has announced "'hat increasing services to the hard- 
to- serve is one of its primary goals for JTPA in the upcoming program year. 
Because the performance standards system has a major influence on who is 
served, it is important for the National Office, the States, and SDAs to 
assess whether the performance standards system promotes the goals of the 
program. Officials at all levels should consider the proposals mentioned 
above and determine if one or more of them might be used to help promote 
their policies. 
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I. Introduction 

In October 1983 the Job Training Partnership Act (JTPA) began 
operation as the country's major employment and training program. JTPA 
replaced the training programs operated under the Comprehensive Employment 
and Training Act of 1973 (CETA) , and aT?A was more than simply a change in 
name only. In addition to shifting a major part of the administrative 
responsibilities from the Federal to the State level of gover JTPA 
also greatly emphasized performance of the programs by including provisions 
for rewards and sanctions for local service delivery areas (SDAs) that 
exceed or fail to meet performance standards established by the Secretary of 
Labor as modified by the Governor. 

The increased emphasis in JTPA on performance, as measured primarily 
by the quality and quantity of placements, has led to concern that JTPA is 
not adequately serving those with the greatest needs for employment and 
training programs. Frequently, the program is accused of "creaming" among 
the eligibles, although the term creaming is rarely concisely defined. ^ The 
concern generally has been that the service delivery areas (SDAs) have 
concentrated their services on eligible individuals who are most likely to 
succeed in the labor market regardless of whether or not they participate in 
the program. If true, this assertion means that some eligible individuals 
who could benefit from JTPA are being denied access to the program, in favor 
of individuals who need the program less and may benefit less from the 
program. 

At the same time, few if any members of the employment and training 
community wish to return to the days when success was measured almost 
exclusively by who was served rather than by the effectiveness of the 
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programs. Although the concept of measuring program performance on the 
basis of outcomes was used to some extent in the later years of the CETA 
program, it is an integral part of JTPA. 

This report has been prepared to help States, SDAs, private industry 
councils (PICs) , service providers, and ethers interested in employment and 
training issues explore how target groups of interest can be identified and 
how the performance standards system can be used to encourage the enrollment 
of hard- to-serva individuals. The term hard-to-serve will be used to refer 
to individuals with especially severe deficiencies or barriers that are 
likely to make them more costly to serve and less likely to find and retain 
employment. Examples of such deficiencies include low reading and math 
capabilities, and little or no work experience. One of the people we spoke 
with suggested that deficiencies be classified in four categories: basic 
skills (reading and math), job-ypecif ic skills (occupational skills), work- 
related skills (e.g., attitudes, appearance, and punctuality) , and job 
search skills. Examples of barriers to employment include lack of day care 
and lack of transportation, 

We distinguish the concept of being hard-to-serve from the related 
concept of being ,f dif f icult- to-place , " even though these concepts are often 
used interchangeably. Diff icult-to-place individuals are defined here as 
individuals in demographic groups that SDAs have generally had greater 
problems placing. For example, older workers, members of minority groups, 
and women do not necessarily have deficiencies that make them hard-to-serve, 
but JTPA and CETA experience shows that members of these groups are 
diff icult-to-place . 

In theory there is not necessarily a conflict between the goals of 
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enrolling the hard- co-serve and encouraging effective programs. In 
practical terms, however, these two program objectives can come into 
conflict. By definition SDAs take risks when they enroll hard-to-serve 
individuals, and if the performance standards system does not include 
adequate adjustments in the level of expected performance for serving these 
individuals, SDAs face a tradeoff between enrolling the hard-to-serve and 
achieving a high level of measured performance. Low performance can result 
in losing performance standards awards and, in extreme cases, in sanctions 
against the SDA. 

In the current JTPA performance standards system there are significant 
costs and other difficulties in trying to develop objective statistical 
adjustment formulas to make the system neutral regarding serving individuals 
with severe labor market problems. The regression modeling approach can 
correct for a limited number of factors, but some of the factors of interest 
may be very difficult and expensive to measure and collect. For example, 
measuring work attitudes is likely to be difficult or impossible. 

Finally, it is worth reconsidering the tradeoff between program 
efficiency and equity of services. ^ The current performance standards 
system is based largely on the premise that the performance standards 
adjustments should hold SDAs harmless for serving individuals with severe 
labor market problems. Lowering standards for the hard-to-serve might: 
provide a sufficient extra incentive, but some PIC officials and others may 
be more concerned about the lower level of performance without taking 
account of adjustments for serving hard-to-serve individuals. In addition, 
consideration should be given to structuring the performance standards 
system so that SDAs are provided extra credit rather than held harmless for 
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serving individuals with severe labor market barriers. 

This guide is intended to help explore these issues. The material 
presented here is based on discussions with a number of representatives from 
the Federal Government, State Governments, SDAs, PICs, academic 
institutions, and others interested in employment and training programs. 
The ideas presented do not necessarily represent the positions of the 
National Commission for Employment; Policy or the U.S. Department of Labor 
unless so indicated; in fact, in the interest of stimulating consideration 
of a variety of ideas, we have raised some ideas that are not supported by 
the Department of Labor. The intent is to stimulate thought and 
consideration of alternative definitions end policies regarding the hard-to- 
serve for State, SDA, and national officials involved in developing and 
implementing Title II-A JTPA programs and policies for economically 
disadvantaged youth and adults. 

The reader should keep in mind the limitations of this study. First, 
this guide focuses on performance standards and the use of JTPA six-percent 
incentive funds as they relate to hard- to -serve individuals. Other features 
of JTPA also affect the extent and types of services provided, but such 
issues are not addressed here. For example, the statutory limits on 
stipends and administrative costs may affect who is served and the nature of 
services provided, but we will not address these issues here. A second 
limitation of this study is that we have focused more on adults than youth. 
Many if not all of the issues discussed here apply to youth as well as 
adults, but we have not addressed the special difficulties faced in serving 
youth . 

It should also be kept in mind that the effort involved in producing 
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this guide was an informal assessment of the views of a number of experts in 
the employment and training field. It was not a large-scale investigation 
of the relationship between performance standards and who is served under 
JTPA. Because the current study is based a nonrandom sample of experts, 
the findings may not be representative of the JTPA system as a v;hole. 

The next section of the guide presents a orief review of the 
performance standards system in JTPA. Section III discusses the concept 
"hard-to-serve 11 and distinguishes it from other targeting issues; this 
section also provides the views of a number of individuals in the JTPA 
system on operational and theoretical considerations involved in defining 
the concept hard- to -serve. Section IV of the report describes methods that 
have been used and others cb c might be considered in promoting services to 
hard- to -serve clients. Finally, Section V presents a summary of the 
findings and conclusions. 

Ijl- Performance Standards in JTPA 

This sections provides a brief overview of performance standards in 
JTPA. The section provides a brief description of performance measures, the 
methodology used to develop the performance standards, and the requirements 
for issuing awards or sanctions determined by performance standards. 

Performance Measures 

JTPA differs from its predecessor, CETA, in that there is a strong 

emphasis on performance outcomes. The law indicates the importance of being 

able to measure return on the investment of JTPA in Section 106: 

...it is essential that criteria for measuring the return on this 
investment be developed and... the basic measures of performance for 
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adult training programs under title II is increase in employment and 
earnings and the reductions in welfare dependency resulting from the 
participation in the program. In order to determine whether these 
basic measures are achieved, the Secretary shall prescribe standards 
on the basis of appropriate" factors which may include (A) placement in 
unsubsidized employment, (B) retention in unsubsidized employment, (C) 
the increase in earnings, including hourly wages, and (D) reduction in 
the number of individuals and families receiving cash welfare payments 
and the amounts of such payments. 



As a *esult of the language in the law, seven performance measures 
(four for adults, three for youth) have been developed. The measures and 
the national standards for PY 84-PY 85 and PY 86-PY 87 are listed below. 

NATIONAL STANDARDS PY 84-PY 85 PY 86-PY 87 

ADULTS 

Entered Employment Rate . 55% 63% 

Average Wage at Placement $4.91 $4.91 

Cost per Entered Employment $5704 $437* 

Welfare Entered Employment Rate 39% 51% 

YOUTH 

Entered Employment Rate 41% 43% 

Positive Termination Rate 82% 75% 

Cost per Positive Termination $4900 $4900 



The Department of Labor is currently collecting data on postprogram outcomes 
and is considering issuing postprogram standards. 

National standards are set by the Department of Labor and are based c l 
SDA performance and national policies. Governors can adjust the standards 
based on local factors. Application of performance standards can be broadly 
described as a three stage process: 

1. Establishment of uniform measures and national performance 
standards by the Department of Labor; 

2. Establishment of additional standards by the Governor if desired 
and optional adjustment of standards using adjustment models developed 
by the Department of Labor for variations in local factors, terminee 
characteristics, and length of services (for cost measures) or other 
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appropriate adjustment procedures developed by the Governor; and 

3. The determination of whether the given SDA displayed superior, 
expected, or substandard performance and providing the appropriate 
bonuses, technical assistance, or sanctions. 

National Performance Standards 

The Secretary of Labor sets national performance standards. The 
standards cannot be modified more than once every two program years, and the 
modifications cannot be retroactive. The performance standards for the 
initial year of JTPA (known as the transition year) were based on CETA data. 
The performance of the CETA program was also used as a guide for setting PY 
1984 and PY 1985 standards since no JTPA data were available. PY 1986 and 
PY 1987 are the first years for which standards were based on actual JTPA 
data. 

Adjustments by the Governor 

Although establishing the national standards is the first step in the 

performance standards process, the step that affects SDAs most directly is 

the Governor's adjustment. Section 106(e) of the law states: 

...Each Governor may prescribe, within parameters established by the 
Secretary, variations in the standards under this subsection based 
upon specific economic, geographic, and demographic factors in the 
State and in service delivery areas within the State, the 
characteristics of the population to be served, and the type of 
services co be provided. 

Basically, a Governor has four options for setting local performance 

standards : 

1. Make no adjustment and use the national standards; 

2. Use the regression based adjustment model as provided by the 
Department of Labor; 
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3. Use the regression based model provided by the Department and make 
further adjustments for factors unique to a given State or SDA; or 

4. Develop alternative adjustment procedures consistent with the 
Secretary of Labor's prescribed parameters. 

Governors may also add additional performance measures to the ones issued by 

the Department of Labor. 

The regression models developed by the Department attempt to estimate 

the relationship between relevant factors (i.e., economic and geographic 

factors in the State, characteristics of the population to be served, and 

types of services to be provided) and their expected effect on the 

performance measures. Governors also have the option of developing their 

own adjustment procedures, but such adjustment procedures must adhere to a 

set of parameters established by the Secretary. ^ 

Incentives and Sanctions 

In order to reinforce the goal of high performance in the JTPA 
program, six percent of the Title II -A funds are allocated to the States to 
provide performance awards, incentives for serving hard-to-serve groups, and 
techrical assistance to SDAs. It is the responsibility of the Governor to 
develop an awards system for the State, and a wide variety of systems have 
been developed. For example, an SDA may be required to meet a certain 
number of the seven standards or a specific standard to be eligible for a 
reward. Most States award money in proportion to the degree by which a 
given SDA exceeded the standard, as is specified in the statute. The awards 
can be presented in a variety of ways, including a bonus to the SDA to be 
used specifically for encouraging services to the hard- to -serve. 

A portion of the six-percent funds may be set aside to provide 
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technical assistance for SDAs that have difficulty meeting the performance 
standards. An SDA that fails to meet performance standards for two 
consecutive years will be i 'ganized at the discretion of the Governor. 

III- Defining the Hard-to-Serve and Other Targeting Issues 

Because employment and training programs have never been able to serve 
more than a small fraction of the individuals who could potentially benefit 
from the programs, an important issue under all major employment and 
training programs has been how much the program should be targeted and on 
what criteria. In addition, once eligibility criteria are established, a 
decision must be made on whether preference should be given to serving 
certain categories of eligible individuals. In this section we first 
discuss references to targeting in the statute and summarize the criteria 
that may be used in selecting target groups. We then discuss tha Department 
of Labor's regression adjustment models as a means of encouraging services 
to hard-to-serve groups. Next we consider additional actions taken by 
several States to encourage services to the hard- to -serve, and finally we 
discuss the findings from a performance standards task force on defining the 
hard-to-serve population in JTPA. 

Statutory Provisions on Targeting 

Participat .on in programs sponsored under Title II -A of JTPA is 
largely limited to individuals who meet the statutory requirements for being 
classified as economically disadvantaged. While there are five criteria in 
the statute for * eting the definition of economically disadvantaged, most 
individuals are eligible because they meet one of the two poverty 
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definitions and/or are recipients of cash welfare payments or food stamps. 4 
While some individuals whose family income exceeds 125 percent of the 
poverty level will be eligible because they are recipients of food stamps, 
it is important to bear in mind that the eligibility requirements under JTPA 
are fairly strict. The only exception to the standard eligibility 
requirements is that up to 10 percent of the participants enrolled by an SDA 
are not required to meet the economically disadvantaged requirement if they 
have some £ther barrier to employment. 5 Thus, when concern is expressed 
about SDAs creaming or serving those who are not most in need, it is 
important to note that almost all individuals eligible for JTPA are 
economically disadvantaged and that anj selection taking place is among 
those that are economically disadvantaged. 

Only a fraction of those eligible for JTPA can be served in a given 
year. Sandell and Rupp (1987) have recently estimated that 12.9 percent of 
the eligible unemployed individuals between the ages of 16 and 64 were 
served in Program Year 1985 (July 1, 1985 - June 30, 1986). 6 The statute 
provides flexibility for SDAs to determine which individuals to serve, but 
there are several requirements for serving particular groups, and States can 
provide further direction to the SDAs. 

The most specific requirement in the statute for serving particular 
individuals is the requirement in Section 203 that at least 40 percent of 
the Title II -A funds be spent on youth ages 16 to 21.^ Section 203 also 
requires that recipients of Aid to Familit. *ith Dependent Children (AFDC) 
and dropouts be served on an equitable basis. 

Several other provisions in JTPA also make reference to the types of 
individuals to be served, but th ese other references provide less specific 
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directions. Section L41(a) of JTPA states that job training plans are to 
provide opportunities "to those who can benefit from, and who are most in 
need of, such opportunities." Most in need can be thought of as those who 
will fare the worst in the labor market if not provided training. Most in 
need can also be defined in terms of having greater financial 
responsibilities. Thus, some SDAs, such as the PIC of Southern Connecticut, 
consider heads of households and teenage parents to be most in need. Note, 
however, that the statute also indicates that services should be provided to 
those who can benefit from the training. Section 141(a) also indicates that 
job training plans are to make efforts to provide equitable services among 
substantial segments of the population. 

States are permitted to use their six-percent funds to provide 
incentive payments to SDAs for serving hard- to -serve individuals as well as 
awards for exceeding performance standvirds and for technical assistance. 
Thi? provision does not require that the hard-to-serve incentive funds be 
tied to performance on the State 1 s performance measures, but as will be 
described below, some States have elected to base the hard-to-serve 
incentive funds at least partly on the basis of performance. 

The statute does not define the terms "hard-to-serve" and "most in 
need." The two terms are sometimes used interchangeably, but we shall use 
the term hard-to-serve to refer to individuals with severe labor market 
deficiencies or barriers. Youth who exhibit symptoms of labor market 
problems are sometimes referred to as being "at risk." Because of their 
lack of experience, the most disadvantaged youth are generally classified by 
their likelihood of experiencing employment problems in the future, rather 
than by their labor force history. 



id 



12 

The reference in Section 141(a) to serving those who can benefit from 
the program recognizes and points to the limitations of JTPA. Given the 
funding level for JTPA and the statutory provisions restricting stipends and 
supportive services, JTPA clearly cannot provide major employment and 
earnings gains to all individuals who are eligible for the program. Some 
individuals may simply need too much remediation for JTPA to be of much 
help . 

Some observers have described the role in selecting participants for 
JTPA to be similar to the triage performed by doctors on a battlefield. 8 
Some individuals cannot be helped because JTPA is unable to provide 
sufficient resources for them; totally illiterate individuals are sometimes 
considered to be in this category. At the other extreme are those 
individuals who are eligible for the program but are like! y to perform 
adequately in the labor market on their own; these people are analogous to 
the injured with minor wounds who will recover on their own. Finally, the 
group in the middle, those whose problems are serious but not so serious 
that they cannot be helped, is the appropriate target group for JTPA. These 
are the individuals who are in need of services and who can benefit from the 
services. An important issue for States and SDAs is deciding where to draw 
the line on who is too seriously disadvantaged to be helped by the program. 
While in theory there is no reason why a person who is totally illiterate 
cannot be served in JTPA, budget constraints, cost performance standards, 
and a desire to serve a large number of people have led some members of the 
employment and training community to argue that JTPA simply cannot serve 
individuals with no reading or math skills 

An important issue, which cannot be considered in depth at this time, 
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is the relationship between hard-to-serve status, performance standards, and 
program impact. An experiment to determine the net impact of JTPA is being 
conducted by the Department of Labor in 15 to 20 SDAs; however, impact 
estimates for JTPA will not be available for at least three years. In 
addition, the results may not apply to all SDAs or relate impact to measures 
of the deficiencies associated with hard-to-serve status.^ 



Summary of Targeting Issues in JTPA 

At this point it is useful to summarize the terms relating to 
targeting that are found in the statute or are helpful in sorting out 
targeting issues in JTPA. The definitions provided below are not official 
Department of Labor definitions, but they should prove useful in discussing 
performance standards and the hard- to -serve. 

Hard- to -serve. We will use this term to identify individuals with labor 
market deficiencies or barriers to employment. As is discussed more below, 
the most common deficiencies are thought to be a lack of basic skills, 
particularly reading skills. Such individuals are likely to require more 
intensive or longer-term services, and the probability of success may be 
lower . 

Difficult- to-place. This term will be used to describe groups for whom 
evidence has shown that SDAs are likely to experience problems in obtaining 
placements, particularly in well -paying jobs. Examples include the 
demographic groups with negative weights in the Department of Labor f s 
regression adjustment models such as women, minorities, *nd welfare 
recipients. The lower placement rates for these groups may result because a 
disproportionate share are hard-to-serve or because of discrimination in the 
labor market. 

Most in need. Individuals may be considered to be among those most in need 
based either on their preprogram level of family income or their 
responsibilities for supporting a family. The determination of a precise 
definition can be made at the State or SDA level. 

Equitable provision of services (EPS) groups. The statute requires that 
services be provided equitably to dropouts and AFDC recipients, and that 
efforts be made to serve "substantial segments of the population" equitably, 
without defining the substantial segments. States and SDAs must develop 
policies on which groups they consider important for meeting this provision. 
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Note* that the EPS provision does not necessarily include only groups that 
are difficult- to-place. 

At Risk youth. This term is used to describe youth who are in danger of 
experiencing employment problems. The term is usually defined by States and 
SDAs to include dropouts, potential dropouts, and youth who have already 
exhibited some signs of developing labor market problems, e.g., teen 
parents, substance abusers, and offenders. 

There are some interesting differences in the terms presented above. 
Note that hard -to -servo refers to characteristics of an individual, but 
difficult-to-place reff -s to characteristics of a group. Individuals who 
are most in need may or may not also be difficult- to-place or hard-to-serve, 
but hard-to-serve individuals will almost always be dif ficult-to-place. 
Also, there may be conflicts ir trying to target on all the groups mentioned 
above .» If a State or SDA interprets the EPS provision as meaning 
proportional representation of all significant segments, this may conflict 
with providing special attention to those most in need or hard-to-serve. 

Although it is possible to develop national definitions for the 
concepts listed above, it Js >ioz necessar Regardless of whether or not 
national definitions are established, States and SDAs need to establish 
their own working definitions and policies to set priorities and negotiate 
the development of appropriate standards and adjustments, particularly if 
State and SDA priorities differ. 

For purposes of the performance standards system, the key factors to 
identify for adjustment models are those that characterize hard-to-serve and 
difficult- to-place status. Indicators of being in either status should be 
included in the models to hold SDAs harmless for serving individuals with 
such characteristics. 
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The Department of Labor f s Adjustment Models and Their Limitations 

The Department of Labor f s regression adjustment models, which form the 
basis for most States 1 performance standards systems, were developed to hold 
SDAs harmless for serving individuals who are difficult- to-place or hard-to- 
serve. The factors included in the models represent terminee 
characteristics and local economic conditions that are associated with the 
performance measures. The factors and models have been developed through an 
extensive consultative process, and the factors that are included represent 
those variables that have a statistically significant effect on tb outcome 
and cost measures. 

The terminee factors in the models should, ideally, include all 
factors that affect the performance measures. To the extent that 
characteristics such as sex, race, and ethnic group are associated with lower 
entered employment rates and wage rates at placement and higher costs of 
services, individuals with these characteristics can be considered 
difficult- to-place relative to participants with different characteristics. 
For example, the negative weights in the adult entered employment rate model 
for females indicates that females are more difficult- to-place than males. 
If the models are correctly specified, then SDAs will be held harmless in 
their performance standards for serving individuals who are likely to have 
lower probabilities of finding a job after participation, lower expected 
wage rates, and higher than average costs of services. 

The tradeoffs implied by the regression adjustment models are not 
always easy to see. To illustrate these tradeoffs, we will consider an 
example using the PY 1987 model for the adult entered employment rate. 
Exhibit 1 contains a sample worksheet for this model from the 1987 
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Exhibit 1 



SAMPLE 

1> _ 



PY 87 JTPA Pe 


r forma nee Standards Wbrl 


sheet 


A. Service Delivery Area's Nare 


B. SCA timber 


C. Performance I*rlcd 
py 87 


D. Type of Standard 
( 1 Plan 

( I Recalculated 


Date 

Calculated 


E. Performance Measure 

Entered Employment Rate (Mult) 











F. LOCAL FACTORS 

1. t ftmale 

2. % Black 

3. % Hispanic 

4. 1 Asian/Pacific Islaraier 

5. % dropout 

6. % Handicapped 


G. SDA FACTOR 
VAUXS 

b9.0 
45.5 
15.0 
5.1 
30.2 
11.4 


H. NATIONAL 
AVERAGES 

i2.4 ■ 

23.8 
7.9 

i.4 

25.0 


I. DIFFERENCE 
(G MINUS H) 

6.2 

n.7 

7.1 

2./ 

5.2 


J. WEIGHTS 

- .020 

- .081 

- .009 

- .022 

- .1*5 


K. EFFECT OF UXAL 
FACTORS ON 
PERFORMANCE 
EXPECTATIONS , 
(I TIMES J) 

- .12 

- l./b 

- .ut> 

- .Ob 

- .yi 


7. C UC Clairant 

8. % welfare Recipient 

9. Uwpioywent Rate 

10. Papulation Density UOOOs/aq.M. ) 

11. Biployee/Resldent Worker Ratio 
Average Annual Earnings In 

12. Miolesale and Retail Trade (000) 


11.1 
42.0 
9.0 

7.0 

Aol.9 
11.3 


9.1 
10.0 
29.8 
8.0 
0.6 

11.8 


/.j 

.2 

l£. 1 

1.0 
6.4 
4.1 

- .5 


- .093 
.644 

" - .J7< 

- .623 

.Hi 

- .078 

- .463 


- ui 
.009 

- J.J/ 

- .b^ 

- .32 
.23 






L* Total 






- 2.26 


(12-19-86) 


M. NATIONAL DEPARTURE POINT 


♦ 62.4 


N. Model-Adjusted Performance 
Level (l ♦ H) 


60.1 


O. Governor's Ad jus burnt 




P. SOA Performsfce Standard 
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Department of Labor Technical Assistance Guide (TAG). In the sample 
worksheet, the SDA's model-adjusted performance level is 6&.1 percent, 
assuming that there is no Governor's adjustment. This would be the SDA's 
performance standard based on the clients it serves and its local economic 
conditions if the Governor makes no additional adjustments to the model. 

To see how the model holds the SDA harmless for changes in its 
clients* characteristics, consider what happens to the standard if the SDA 
serves 100 percent velfats clients. In that case, the SDA factor value in 
Line 8 would be 100, and the nodel- adjusted performance level would be 44.1 
percent. (This calculation assumes that none of the other client 
characteristics change; this is obviously unrealistic but it does not affect 
the basic point.) On the other hand, if the SDA served no welfare 
recipients (and again kept all other characteristics the same) , the model- 
adjusted performance level would be 71.7 percent. Put another way, the 
mo del -adjusted entered employment rate performance level for this SDA is a 
weighted average of the 44.1 percent for welfare recipients and 71,7 percent 
for individuals not on welfare. Since this result is clearly not obvious 
from looking at the factor weight of -.276, SDAs should compute their model- 
adjusted performance levels under different assumptions about client mix. 
Many States provide computer disks for SDAs to simulate their performance 
standards with different client mixes. 

The regression model adjustment procedure is clearly one method of 
attempting to avoid penalizing SDAs for serving clients with labor marked 
problems. The models have several limitations, and the Department of Labor 
has^ encouraged States to consider additional adjustments. 

The major problem with the adjustment models is that they dc not 
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include all factors that affect the performance measures. Anyone familiar 
with employment and training programs recognizes that the brief list of 
demographic and labor market experience characteristics in the models could 
easily be .supplemented. Omitting relevant factors from the models has two 
important effects. First, rhe implied weight of any factor not included in 
the model is zero. For example, if tne entered employment rate is expected 
to be lower for felons and no variable is included in the 'nodel for percent 
of terminees who are felons, SDAs will not be held harmless for serving 
felons. Second, when factors that belong in the model are omitted, the 
estimated weights of included factors that are correlated with the omitted 
factors are biased. For example, if the variable for percent welfare 
recipients were omitted from the model, the estimated weight for percent 
females served would be biased. The likely effect would be to lower model- 
adjusted performance for women in general and to underadjust for providing 
service * ^are recipients. SDAs v;ould have an incentive to overserve 

nonwe . women and to underserve welfare women. 

SDAs interested in maximizing their measured performance might respond 
by underserving individuals with characteristics associated with low 
performance but not included in the adjustment models. Thus, one possible 
reason that the estimated weights in the regression adjustment models have 
become smaller in successive years is that SDAs have become more adept at 
selecting participants on the basis of factor? that are omitted from the 
models. 

The*** are a mimber of reasons why not all relevant factors are 
included ir :* regression adjustment models. First, there is not a 
consensus as > what all the relevant factors are. Second, there are both 
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practical and statistical reasons to limit the number of factors included in 
the models. On the practical side, adding additional factors would add to 
the data collection burden for SDAs L*id make the required computations more 
complex. Statistical problems might result when factors are added because 
if several factors are highly correlated, then the individual weights may be 
less precise and change significantly from year to year. Major changes in 
the weights affect the incentives to SDAs and makes the system less stable. 
A third problem with adding more factors is that some factors may be 
difficult or impossible to measure.^ Some factors, such as work attitudes, 
are very hard to define, and no good measures are currently available. 

There are several other limitations to the regression adjustment 
models. For example, it is possible that the factors and weights may vary 
by region, and it is possible that the models should include "interaction 
terms" (products of two or more variables) . ^ These issues have been 
explored by the States in Region V, and their preliminary results are 
reported below. 

The factors included in the current set of Department of Labor 
adjustment models include a number of diff icult-to-place factors, but few 
factors that could be considered to measure hard-to-serve status. In 
particular, there are no indicators of basic skills deficiencies or lack of 
work experience, which many observers believe are the key deficiencies that 
make a person hard- to -serve. The Department of Labor recognizes these 
limitations, and for PY 1988 the Department of Labor has proposed collecting 
information in the JTPA Annual Status Report (JASR) on long-term welfare 
status, minimal work experience, and reading level deficiencies, which are 
all potential measures of labor market deficiencies. 
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The Department of Labor's 1987 Technical Assistance Guide includes 
instructions on how States can add additional factors for making adjustments 
to their performance standards. The TAG recognizes that in some instances 
adjustments are acceptable without full knowledge of the extent to which 
performance is expected to be affected by serving individuals with certain 
characteristics. In such circumstances the TAG recommends using a 
reasonable estimate until data can be collected. 

State Policies on Defining the Hard-to-Serve 

States have several options available to promote the provision of 
services to groups that they consider hard-to-serve. It is important to 
recognize that simply designating a group as hard-to-serve does not 
necessarily assure that incentives will be provided to serve members of the 
designated group, and incentives can be provided to serve groups that are 
not formally designated as being hard-to-serve. For example, if the State 
sets aside a very small pool of funds for serving the designated groups, the 
incentive will be quite weak. On the other hand, States can undertake 
several actions to encourage seivices to particular groups without formally 
designating them as hard- to -serve. Increasing the size of the weights in 
the models for specific groups, e.g., minorities, directly changes the 
incentives for serving these groups, but such changes are not encouraged by 
the Department of Labor and have been used sparingly. 

The strength of incentives to serve designated target groups is a 
function of the amount of funds that can be gained by SDAs for serving these 
groups and the difficulty in meeting the award criteria. It is important to 
keep in mind that because most of the Title II-A JTPA funds, 78 percent, are 
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distributed by formula to the SDAs, States may have a greater impact on SDA 
behavior by influencing how the 78-percent funds are spent than on how the 
six-percent funds are spent. 

The State of Wisconsin conducted a survey of 20 States in 1986 to 
assist the State in establishing its six-percent policies. As part of the 
survey, the Wisconsin researchers determined which States had established 
"incentive payment formulas" for serving designated hard-to-serve groups. A 
summary of their findings is provided in Exhibit 2. The groups designated 
as hard- to- serve include welfare recipients or specific categories of 
welfare recipients (4 States), dropouts (3 States), minorities (3 States), 
females (2 States), oleer workers (1 State), and handicapped workers (1 
State). Some States did not designate any groups as hard-to-serve, and 
Wisconsin designated all six groups listed above as hard-to-serve. 

As we noted above, concern has been growing in the employment and 
training community that many of these variables, while useful for 
encouraging services to dif f icult- to-place groups, do not capture the 
deficiencies that make individuals hard- tro- serve. That is, the lower 
entered employment rates and wages at termination for minority group members 
and females may in part reflect deficiencies and barriers disproportionately 
faced by members of these groups. ^ Several States have recently taken 
actions to go beyond the common demographic and other traditional measures 
to get improved measures of hard-to-serve status. We will review the recent 
efforts of three such States, Michigan, Colorado, and Illinois, here. 

The Michigan Job Training Coordinating Council (MJTCC) otablished a 
Hard- to -Serve Task Force co "study options that address the training needs 
of hard- to -serve persons and to develop recommendations that enhance their 
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Exhibit 2 

Groups Designated as Hard-to-serve in Wisconsin Survey of 20 States 

Welfare Recipients 
Arizona 
Indiana 
Pennsylvania 
Wisconsin 

Dropouts 

Indiana 

Massachusetts 

Wisconsin 

Minorities 

Kansas 

Massachusetts 
Wisconsin 

Females 

Kansas 
Wisconsin 

Older Workers 

Wisconsin 



Handicapped Worker s 
Wisconsin 
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entry into the labor force." The MJTCC's definition of hard- to -serve status 
required that an individual be a dropout, a teenage parent, an offender, or 
have a drug/alcohol problem and either meet another of the foregoing 
criteria or be one of the following: a youth, a public assistance 
recipient, handicapped, a displaced homemaker, a member of a minority group, 
a person with limited English proficiency, an older worker, a person with no 
work experience, or a person unemployed for longer than one year. 

The Michigan task force concluded that the MJTCC definition was 
confusing because of its complexity, and they did not agree with the 
categorical nature of the definition. Their concern with the categorical 
nature of the definition was that the definition did not relate to the 
specific labor market problems faced by the individuals in question and, 
therefore, was not useful in developing specific employment and training 
programs for such individuals. The task force recommended that four 
principles be used in developing a definition of hard-to-serve: 

1. The definition should be inclusive rather than exclusive; that 
is, individuals who are unemployed and experiencing multiple 
barriers to entry into the labor market should be eligible for 
special services. 

2. The definition should be functional, apply across programs, and 
lead to measurable outcomes . 

3. The definition should recognize that employment and training 
programs for the hard-to-serve generally will last longer and 
will be more costly than typical programs. 

4. Functional illiteracy should be identified as one of the major 
barriers to employing the hard-to-serve and should be reflected 
in the definition. 

Based on these principles, the Michigan task force recommended the following 

definition for h<ird- to-serve : 

A person rhall be considered "hard- to-serve" who is 
economically disadvantaged; unemployed for two or more years or never 
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been employed; in need of functional literacy or employability 
characteristics or supportive services; and whose employability 
development plan states that the length of time or cost of training 
leading to employment exceeds that received by the average person 
enrolled in a designated area. 

The Michigan task force's proposed definition of hard-to-serve differs 
significantly from definitions based on demographic and other categorical 
characteristics. Basing the definition on the quantity of services to be 
provided could be questioned because it permits providers to define hard-to- 
serve status on services received rather than need, but the strict lack of 
employment criterion assures that only individuals with strong needs will be 
included. 

Colorado used an interesting approach in PY 1984 and PY 1985 to 
encourage SDAs to serve groups of interest while maintaining local 
flexibility. The State established six target groups of interest and 
required that SDAs meet entered employment rate standards for two groups 
(dropouts and offenders). For the other six groups (displaced homemakers, 
AFDC recipients, handicapped, teenage parents, dislocated workers, and older 
workers), SDAs had to meet standards for any two of the six groups. Because 
the populations and priorities often vary within a particular State, 
Colorado's approach of providing some flexibility may be useful for other 
States to consider. 

Illinois, along with the other States in Region V, has been engaged in 
a major effort to refine the regression adjustment models. The Region V 
States retained Northern Illinois University (NIU) to assist them in 
developing regional and State models making use of individual as well as 
SDA- level data. Each of the States in the region has worked with NIU to 
develop State -lr/el and regional models. At this time, Illinois is the only 
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State that has developed final models. The Illinois models are tc be tested 
and refined during PY 1987, with implementation anticipated in PY 1988. 

The Illinois effort was directed toward improving the adjustment 
models in several ways; improving services to hard-to-serve clients was only 
one of the goals. Of particular concern to Illinois is the omission of 
activity variables, e.g., type of classroom training and long-term training, 
in the Department of Labor adjustment models. The Illinois staff and NIU 
researchers believe that individuals who are hard- to -serve often require 
long-term training and reading and math remediation, and that the omission 
of these factors from the Department of Labor models discourages services to 
individuals who need these services. Thus, the Illinois models include 
activity variables; as expected, the activities that are used primarily for 
individuals with the greatest deficiencies (e.g., classroom training greater 
than 26 weeks) have negative weights. 

The counter- argument to this position is that activity assignment is a 
management decision; SDAs should assign people to whatever activity is most 
helpful for them. By including activities in the models, SDAs might begin 
assigning individuals who do not require extensive remediation to these 
activities simply to lower their model-adjusted performance standard. 
According to the counter -argument, the better way to provide incentives for 
serving those with severe deficiencies and barriers is to include better 
measures of hard-to-serve status in the models. 

The Illinois models also include several additional categories of 
clients in various models. These categories are offenders, displaced 
homemakers, rural clients, and teen parents. The Illinois models are 
included as Appendix 1 to this report. 
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Findings of the POL Hard- to -Serve Task Force 

Improving targeting to hard-to-serve individuals is one of the 
priorities of the Department of Labor for developing performance standards 
in PY 1988. The Department of Labor established a Hard- to-Serve Task Force 
as part of the Performance Standards Technical Work Group to assist the 
Department in analyzing the attributes of the hard-to-serve population. The 
task force was co-chaired by Dr. Linda Odum and Mary Reid of Virginia and 
included representatives from New York, Wisconsin, California, and 
Massachusetts. Because of the tight schedule for the performance standards 
development process, the task force was required to complete its work within 
one month. Members of the task force used a variety of techniques to gather 
information. Some discussed the issues with their colleagues in the State, 
while others conducted surveys of SDAs and contractors in their States. The 
Massachusetts representative sent a mail survey to SDAs in the New England 
region. The members of the task force sent their findings to Dr. Odum and 
Ms. Reid, and they prepared a summary of the findings. 

Task force members were asked to provide information that could be 
used to define hard-to-serve status. They were asked to categorize the 
characteristics as either deficiencies or barriers. The term deficiencies 
refers to characteristics of individuals that are expected to impede their 
labor market success; examples include lack of vocational skills and 
illiteracy. Barriers refer to environmental characteristics that hinder 
success in the labor market; examples of barriers include a lack of 
transportation and lack of child care. Many of the respondents did not 
differentiate between barriers and deficiencies. Also, many respondents 

o 34 
ERIC 



27 

included target groups that are difficult- to-place in their responses. 

As might be expected in such an effort, there was a wide range of 

responses. The most common responses, as summarized by Dr. Odum, were: 

. Low reading level 

. Low math level 

. No or minimal work history 

. L»ck of daycare 

. Ex-offenders 

. Long-term welfare recipients 

. Lack of public and private transportation 

. Handicap (physical, mental, or emotional) 

. Substance abuse 

. Poor attitudes 

. Parent (single, female, or teen) with child under six 
We have analyzed t^e responses and categorized them as deficiencies, 
barriers, or target groups. In some cases, we may have considered a 
characteristic a deficiency where others might classify it as a barrier and 
vice versa. A complete listing of the characteristics of the hard- to-eerve 
identified by the task force is provided in Exhibit 3. 

Some interesting patterns emerged from the task force report. First, 
although respondents were not asked to identify target groups, most 
respondents listed at least one target group, and five target groups were 
among the most commonly cited characteristics. In addition, the target 
groups taken together probably account for a majority of JTPA participants. 
A definition of hard-to-serve that includes most JTPA participants does not 
accomplish the goal of focusing on the most disadvantaged clients. 

The deficiencies in the list cover a variety of problems that are 
likely to affect labor market success. One respondent suggested that the 
deficiencies could be classified in three categories: employability , 
trainability , and dependability. One of the people we spoke with suggested 
that the deficiencies be classified in four categories: basic skills 
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Exhibit 3 

Characteristics of the Hard-to-Serve Population 
Identified by the Department of Labor Task Force 



Deficiencies 

Low reading level* 

Low math level* 

No or little work 
experience* 

Bad attitudes* 

Lack of good counsel 

Lack of hope 

Health problems 

Lack of work skills 

Long-term unemployment 

Lack of communication 
skills 

Lack of interpersonal 
skills 

Limited English 

Lack of problem- 
solving skills 

Poor work ethic 

Unrealistic job expectations 

Work readiness skills 

Poor appearance 



Barriers 
Transportation 

(public or private)* 
Day care* 
Low income 
Lack of credentials 
Excessive employer demands 
Geographic isolation 
No telephone 



Target Groups 
Ex-Offenders* 
Long-term welfare* 
Handicap (physical, 

mental, emotional)* 
Substance abuse* 
Parents with 

children under 6* 
Youth 
Minorities 
Older workers 
Veterans 

Displaced homemakers 
More than 3 children 
Learning Disabled 
Homeless 

Head of household 
Battered women 



Note: * indicates most common characteristics identified. Other 
characteristics were listed by one or more respondents. 



Source: ICF analysis of Department of Labor Hard-to-Serve Task Force Report. 
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(reading and math), job-specific skills (occupational skills), work-related 
skills (e.g., attitudes, appearance, and punctuality), and job search 
skills. 

The PIC of Southern Connecticut provided the task force with a scoring 
system it uses to rank clients on their hard-to-serve status, and a copy of 
this scoring system is provided in Exhibit 4. When they had more applicants 
than they could serve, they used the scoring system to ration slots. Note 
that in this instance the PIC placed heavy emphasis on most-in-need measures 
such as head of household and teenage parent; an alternative scoring system 
could be devised to emphasize the hard-to-serve. 

Several respondents objected to the concept of developing a national 
definition of hard-to-serve even though the Department of Labor has not 
proposed developing such a definition. One argument is that setting target 
group priorities should be at the local level. Another point raised was 
that designating some individuals as hard-to-serve implies that the other 
participants are somehow easy to serve; once particular groups are 
designated as hard- to- serve, charges of creaming might increase. Some 
respondents also noted that decisions on which groups would be classified as 
hard-to-serve would be made on a political basis rather than on the basis of 
need. One of the individuals opposed to establishing a national definition 
of hard-to-serve groups indicated that he was not opposed to adding model 
adjustments to compensate SDAs for the lower expected success from serving 
individuals with barriers, but he objected to the Department of Labor using 
the designation hard-to-serve for selected target groups. While these 
concerns were expressed by a minority of the respondents to the task force, 
there are undoubtedly others in the employment and training community with 
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EXHIBIT A 



TARGET POINT ASSESSMENT 



Applicant Name 


Social Security Number 




SOCIAL CHARACTERISTICS 
& 

EMPLOYMENT BARRIERS 


ECONOMIC STATUS 


.6 

Receiving 

Public 

Assistance 


.3 

Other 

Economic 

Disadvantaged 


.1 

Non 

Econonmic 
Disadvantaged 


HEADS OF HOUSEHOLDS 


6 


6 


6 


TEENAGE PARENT 


6 


6 


6 


HANDICAPPED/DISABLED 


5 


5 


5 


MINORITY 


A 


A 


A 


WOMEN 


A 


A 


A 


OFFENDER 


3 


3 


3 


SUBSTANCE ABUSER 
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strong feelings on this issue. It should be kept in mind that if methods 
are not implemented to make adjustments for labor market deficiencies, SDAs 
who wish to serve the hard- to -serve will not be properly assessed when their 
performance is computed. 

The mail survey conducted in New England yielded encouraging findings 

9 

on measuring some of the most mentioned deficiencies. Most of the SDAs that 
responded collect information on the math and reading skill levels of 
participants. However, the SDAs use a number of different tests, and there 
is a wide range of responses regarding what grade level makes an individual 
hard- to -serve. Some respondents to the task force consider anyone with less 
than a twelfth grade level of reading and math skills deficient, while 
others believe that fourth grade is an appropriate cutoff. 

The New England survey also found that SDAs generally tracked work 
histories, but none of the SDAs tested for work attitudes. Thus, it 
probably is not difficult to add measures of work history deficiencies to 
the adjustment models. 

IV- Policies to Encourage Services to Hard-W- Serve Individuals 

In this section we present policy options that can be considered to 
encourage services to hard-to-serve individuals. Some of these options must 
be actively considered by States in developing their annual plans. Tor 
example, States must decide how to allocate their six-percent funds. Other 
options require special consideration and may require additional data 
collection and analysis. An example of this type of policy is the 
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modification of the national adjustment models. These latter changes can be 
done at either the national or State level. 

In assessing various options to encourage services to hard-to-serve 
individuals, it is important to recognize that policies that encourage 
serving the hard-to-serve may conflict with some of the other goals of JTPA. 
For example, in States that establish fixed amounts for the uses of six- 
percent funds (rewards for superior performance, incentive funds for serving 
tL^ hard-to-serve, and technical assistance), policies that allocate a 
greater share of the six-percent funds to serving hard-to-serve individuals 
will diminish the funds available for rewarding superior performance and 
technical assistance. Not all policies that encourage services to the i.ard- 
to-serve will require compromises with other program goals, but when they do 
it is important to be aware of the tradeoffs involved. 

In addition to affecting incentives for high performance, we are also 
concerned with the extra burdei, that may b-* placed on SDAs and providers by 
some policies and possible interference with State and local policy 
discretion. For example, adding additional factors to the adjustment models 
may requir^ additional data collection. On the other hand, we have noted 
that many SDAs already collect information on work histories and basic 
skills, two of the most important deficiencies discussed. As we discuss the 
policy options available, we will note the major conflicts that may arise. 

The policies discussed below have been grouped in two categories: (1) 
policies for reducing disincentives to serve the hard-to-serve, and (2) 
policies for providing incentives to serve this population. 
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A. Policies to Reduce Disincentives to Serve the Hard- to-Serve 

As discussed above, certain aspects of the JTPA performance standards 
system may create disincentives for SDAs and their service providers to 
enroll hard-to-serve individuals. The options discussed below may reduce or 
eliminate such disincentives. 

1. Adding factors to the models for deficiencies and barriers . 
Virtually everyone consulted in the course of t h is study stated that 
demographic characteristics do not adequately measure how hard-to-serve an 
individual is. There was also a gener,^ consensus that the key deficiencies 
are a lack of reading and math skills and little or no work experierce. 
Modifying the adjustment models to include measures of these deficiencies is 
feasible at the national level, and individual States could also undertake 
modeling efforts. The Department of Labor has proposed adding reading level 
data and information on minimal work history to the JASR. 

Adding measures of lack of work experience should be relatively easy 
since the information can be obtained at application, and many SDAs already 
collect such data. Measures of prior work experience can be (and frequently 
are) collected for JTPA enrollees. At the national level, the Job Training 
Quarterly Survey (JTQS) contains some information on prior work experience, 
and analyses could be conducted to determine how alternative measures are 
related to labor market experiences after participation. Although the data 
from the Continuous Longitudinal Manpower Survey (CLMS) that was used to 
evaluate CETA programs are dated, the CLMS included data from social 
security records that indicates years in which participants had positive 
covered earnings. The Department of Labor could conduct exploratory 
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research on these data sets to identify promising measures and develop 
estimates of the magnitudes of the weights. 

States should also consider conducting similar exploratory research. 
In the survey of New England SDAs conducted for the Hard-to-Serve Task 
Force, all responding SDAs indicated that they already collect enough 
information on prior work history to ascertain if participants had little or 
no prior work experience. Another potential source of prior work history 
data is 'unemployment insurance records. States vary significantly in how 
accessible such data are and in how long the data are retained, but some 
States could make use of such data. 

Analyses are needed to determine the best "lack of work experience 11 
deficiency measure. Most of the individuals consulted believe that an 
appropriate measure would certainly cover more than 15 weeks of 
unemployment. Some individuals felt that this measure should cover only 
individuals who have never had a job or have not worked in at least five 
years. This issue can only be resolved by considering and analyzing 
appropriate data. The Department of Labor has proposed to add a measure of 
minimal work history to the JASR in PY 1983. The Department of Labor has 
defined a person with minimal work history as "an adult or youth who did not 
work for the same employer for longer than three consecutive months in the 
five years prior to JTPA eligibility determination." 

Long-term AFDC recipients are also often considered hard-to-serve. 
The Department of Labor has defined a long-term AFDC recipient as "an adult 
or youth welfare recipient who had received or whose family had received 
cash payments under AFDC for 24 or more of the 30 months prior to JTPA 
eligibility determination." 
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Adding factors to correct for educational deficiencies is somewhat 
complicated for several reasons. First, educational deficiency data are not 
collected by all SDAs, and it may not be collected for all participants in 
SDAs that do collect such data. Second, there are a large number of tests 
available, and it may be difficult to determine which ones are appropriate. 
Third, there is a wide range of opinion as to what levels of educational 
deficiency constitute a serious deficit and whether "grade level" references 
are appropriate. In the Hard- to -Serve Task Force's effort, the level of 
minimum basic skills required to not be considered hard-to-serve ranged from 
a third grade level to a twelfth grade level. Finally, some testing 
instruments require specially trained administrators and may take n long 
time to give and score. 

While all of these concerns are valid, none of them are serious enough 
to rule out exploration of this area. Requiring testing of all or many JTPA 
participants would add some burden on SDAs and providers that do not already 
test participants as part of the assessment process, so consideration should 
be given to selecting .Instruments that minimize this burden. The Job Corps 
includes testing in Vzs performance standards system, so it is certainly 
feasible. If the reporting element were set as the percentage of 
participants below a certain grade level equivalent (as has been proposed), 
SDAs would not have to administer tests to participants who clearly are 
above that level. It is also possible to permit use of several tests that 
present scores in a common metric, such as grade level equivalent; in that 
way SDAs would maintain flexibility in deciding which tests to use and still 
be held harmless for serving individuals with educational deficiencies. 
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The Department of Labor recognizes the concern in the employment and 
training community about the difficulties of serving people with educatio al 
deficiencies under the current performance standards system. States should 
also consider negotiating adjustment procedures with their SDAs. In States 
where SDAs collect information on reading and math skills as part of the 
assessment process and report the information to the State, the Department 
of Labor's 1987 TAG suggests how States could develop adjustment weights to 
the models by analyzing the experiences of participants with different 
levels of reading and math skills . 

Exhibit 3 listed a number of other deficiencies identified by the 
Hard-to-Serve Tar>k Force. Some of these deficiencies, such as limited 
English, are likely to be relatively concentrated in some States and SDAs. 
In these instances, the affected SDks should negotiate with the State to 
develop an appropriate adjustment. Many of the factors identified by the 
Task Force are probably too difficult to measure and include in the 
adjustment models. Problems such as bad attitudes, lack of social skills, 
and poor appearance are deficiencies, they n.ay not be as feasible to 

address at this time. 

The Task Force also identified lack of transportation and day care as 
common barriers that maks individuals hard- to- serve. These particular 
barriers are more likely to affect cost than outcomes, but they could also 
affect outcomes. Consideration should be given to collecting information on 
the proportion of clients with such barriers and including these factors in 
the cost and outcome adjvistment models. The major argument against such a 
policy is that one of the goals of JTPA is to direct more of the funds into 
training, but the overall cost limitations on nontraining costs may serve as 
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an adequate safeguard. The statute also permits SDAs to apply for waivers 
for increased costs of supportive services. These procedures have not been 
used extensively, but States and SDAs should consider making additional use 
of this provision. 

2. Adding additional target groups to the adjustment models . It is 
interesting to note that many of the responses received by the Hard- to - 
Serve Task Force indicated target groups rather than just deficiencies and 
barriers that were requested. Some of the potential target groups 
identified may not be served in sufficient numbers to be added to the 
adjustment models. The Region V modeling effort is analyzing the use of a 
number of additional target group variables that are available in th. 
States' management information systems. Illinois has found that offenders, 
teenage parents, rural clients, and displaced homemakers are groups with 
statistically significant weights in their models. Other States should 
consider conducting similar analyses. However, it should be kept in mind 
that many people believe that identifying the hard-to-serve on the basis of 
demographic characteristics is not as useful as identifying the underlying 
deficiencies and barriers. 

3. Adding adjustments for activities . Activity variables can be 
added to - regression adjustment models as a method of encouraging 

vices to individuals with labor market deficiencies.^^ The primary 
s toiment for including activity variables in the models is that individuals 
in need of basic skills training and long-term training face greater 
barriers to finding jobs, especially well -paying jobs, and the more 
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intensive training for such individuals is likely to cost more. Because the 
models do not contain adequate measures of these deficiencies, the models do 
not properly adjust the standards to reflect the difficulties of 
participants enrolled in such activities. The most common argument against 
including activity variables in the models is that the decision on what 
activity a person receives is a management decision of the SDA, and the 
models are not intended to hold SDAs harmless from their management 
decisions . 

Both the arguments in favor of including activity variables and those 
against it have some merit. Because measures of basic skills and work 
history deficiencies are not included in the models, the current models are 
likely to penalize SDAs who serve such people; adding activity variables 
will help to overcome this problem. On the other hand, SDAs would receive 
the benefit of a lower standard for all individuals in these activities, 
regardless of whether or not they have deficiencies. 

Until measures of deficiencies are included in the models, States 
should give serious consideration to including activity variables that 
reflect services to individuals with deficiencies. Before adding activity 
variables, States should make sure that consistent definitions of the 
activities are developed. The Department of Labor's TAG provides guidance 
on how activity variables can be added to the models. 

4. Reward structure for exceeding standards . States have a great 
deal of latitude in how they reward SDAs for the extent that standards are 
exceeded. When the extent to which standards are exceeded plays a major 
role in deteimining the size of awards, performance is emphasized relative 



39 

to encouraging services to the hard- to -serve . If SDAs receive the same 
amount of extra tunding for each percentage point they exceed the standards 
they have a strong incentive to beat the standards by as much as possible. 
If, on the other hand, the standard is capped or the excra rewards are at a 
reduced rate, they do not have as strong a financial incentive to get the 
entered employment rate and wage rate at placement to the highest levels. 
With less financial pressure, the SDAs would be able to concentrate more on 
the hard- to -serve. Some States, such as Texas, already reward incremental 
improvements above the standard at a reduced rate. The Department of Labor 
has proposed, as part of its proactive policy guidance, to encourage the use 
of caps in performance incentive policies. 

Another way of using the reward structure to encourage services to the 
hard-to-serve is to give little or no weight to performance on the cost 
measure in determining six-percent rewards. States have great flexibility 
in determining the relative importance of standards, so by minimizing the 
importance of the cost standard SDAs can concentrate more on quality of 
services rather than costs. 

5. The use of Governor's adjustments . Governors are permitted to 
lower the minimum acceptable performance on standards for a variety or 
reasons including encouraging ervices to hard- to -serve individuals. These 
adjustments may be made in addition to or instead of using the models. If 
such adjustments are made, SDAs are less likely to fail to meet the 
standards if they serve individuals who are less likely to gain employment. 
Note that in some circumstances adding tolerance levels to determine if a 
standard has been met only avoids negative incentives for SDAs whose 
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performance is near the sanctioning level. If performance bonuses are based 
on the degree to which standards are exceeded, SDAs that are competing for 
the awards will not be affected in a major way. 

6. Encouraging services through performance standards exemptions . 
Many State and SDA staff have indicated that they have been discouraged from 
developing special programs for individuals with major deficiencies and from 
trying innovative untested activities because of the Department of Labors 
former policy that required such programs to be included in the performance 
standards calculations. In PY 1988 the Department of Labor will allow 
States to determine whether or not the standards apply to individuals served 
and funds expended under incentive awards. (The characteristics of 
individuals served and the funds expended must still be reported on the 
JASR.) States may wish to consider making use of this exemption to 
encourage services to individuals that are especially hard-to-serve or in 
instances where innovative, untested training approaches are used. 

7. Regional or State modeling . Regions or States can develop their 
own adjustment models to improve targeting of the hard- to- serve as well as 
for other purposes. The ongoing Region V effort discussed above has enabled 
the States in the region to obtain models with coefficients specific to the 
region as a whole and for individual States. For demographic factors, there 
may be substantial differences in discrimination and labor market 
opportunities across the country, and regional modeling can be used to 
develop more accurate factor weights. For example, in the case of the 
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entered employment rate for adults, the national mode? has a weight of - 
.081 for blacks, and the Illinois model has a weight of -.090.-^ As noted 
previously, the Illinois effort led to the inclusion of several additional 
factors in the models. The additional terminee factors include displaced 
homemaker status, offender status, and teenage arent status. The Region V 
modeling effort made use of individual participant level data, so it was 
also possible to test the importance of adding "interaction terms" to the 
model. The Illinois pilot model does not include interaction terms, but 
other State or regional efforts may find them important. 

State or regional modeling can also be used to obtain more accurate 
adjustments for factors not included in the national models. The regression 
analysis determines the appropriate weight when other factors are held 
constant, and without developing a model, States must estimate the 
appropriate weight for adjustment factors that are added. 

B. Policies to Add Incentives to Serve the Hard-to-berve 

The policies described below can be used to provide incentives for 
serving the hard- to- serve . They can be used in combination with one or more 
of the policies described above. 

1. The usft of six-percent funds . Six-percent funds may be used for 
performance awards, technical assistance, and incentive payments for se./ing 
hard-to-serve individuals. States that use a greater share of these funds 
to reward performance and less to encourage services to hard- to-i5erve 
individuals provide a relatively stronger incentive for emphasizing 
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performance over services to the hard-to-serve if other things are equal. 
To emphasize the use of six-percent funds for the hard -to -serve, States 
should clearly specify what awards will be made for various levels of 
services to the targeted hard-to-serve participants. 

2. Adding s tandards for the hard-to-serve . A clear way for States to 
encourage opportunities for the hard- to -serve is to include performance 
standards for hard-to-serve groups. The statute specifically states that 
six-percent funds may be used to provide incentives for serving hard- to - 
serve individuals, and many States have implemented such standards. As was 
noted above, the strength of such incentives <* pends partly on the share of 
six-percent funds that are used for such incentive payments. 

Several States have developed standards that tie the hard-to-serve 
incentive funds to outcomes for the designated groups. Such an approach 
encourages performance as well as the provision of services to the 
designated groups. The procedures usea by Massachusetts and Wisconsin are 
reviewed t^low. While the applications in these two States relate more to 
groups that might be designated difficult- to-place rather than hard-to- 
serve, thr principles can be used for other groups. 

Massachusetts han issued two hard-to-serve standards for incentive 
awards in PY 1987. The awards are made for serving at least specified 
percentages of adult dropouts and minorities and achieving acceptable levels 
of performance for these groups in their placement rates and placement 
wages. SDAs receive 2 points if dropouts comprise 30 to 35 percent f all 
out-of-school participants, 4 points if they comprise 35 to 40 percent, and 
10 points if they comprise over 40 percent of all out-of-school 
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partic .pants. SDAs receive 1 point if the placement rate for dropouts is 
within the confidence interval (tolerance range) of the standard for all 
participants, 3 points if the placement rate for dropouts exceeds the 
confidence interval but is less than 110 percent of the standard, and 5 
points if the placement rate for dropouts is equal to or exceeds 110 percent 
of the „ndard. Points fc the placement wage for dropouts are awarded in 
the same manner as for the placement rate. To receive an incentive award, 
an SDA muse score ac least one point in each of the three categories. The 
points are then rescaled as "shares," and the incentive grants are 
determined by dividing the total funds set aside by the total number of 
shares earned and paying each SDA an award for each share earned. The 
incentive awards for serving minorities are determined in a similar manner, 
except that the service levels required are adjusted for the proportion of 
minorities residing in the SDA. Massachusetts has set aside $500,000 for 
hard-to-serve grants and $700,000 for performance grants. 

Wisconsin has used several different performance standards systems in 
recent years, and their approach in PY 1985 was very strongly tied to 
serving six target groups (women, minorities, older individuals, 
handicapped, dropouts, and AFDC recipients). In that year, SDAs had to 
enroll and place all the targeted populations at a minimum of 80 percent of 
their estimated incidence in the SDA f s eligible population to receive any 
six-percent funds. This requirement is the strongest targeting requirement 
that we are aware of. Wisconsin not only required minimum service levels, 
but also awarded part of the six-percent funds to SDAs enrolling and placing 
all six target groups at their incidence in the SDA population. In PY 1986, 
Wisconsin adopted a policy that placed less emphasis on the hard-to-serve: 
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SDAs that failed to serve at least 80 percent of the proportional share for 
the six target groups were to receive only one- third of the maximum 
performance bonus if they otherwise qualified for such a bonus. The 
Department of Labor cautioned Wisconsin that their PY 1907 system might 
overly diminish the importance of the Secretary's standards because SDAs 
that exceeded the Secretary's standards received limited awards; Wisconsin 
then dropped the requirement that the six target groups be adequately served 
to receive performance bonuses. 

3. Giving e x tra credit for the hard-to-serve . Another method of 
encouraging SDAs to serve hard-to-serve individuals is to award extra credit 
for positive outcomes for members of selected groups. For example, in 
computing performance on the adult entered employment rate measure, an SDA 
would receive credit for 1.5 or 2.0 people placed if the person is a member 
of a designated hard-to-serve group. 

To illustrate how such a system would work, consider the following 
example. Suppose that a State has agreed to grant double placement credit 
for long-term welfare recipients. Further suppose that the chart 'below 
gives the information on the participants and their outcomes: 



Long-Term Welfare 



Other Terminees 



Total 



Number terminated 



200 



800 



1000 



Entered Employment 



60 



600 



660 



In this example, the entered employment rate without an extra credit 
system would be 66 percent ([60 + 600]/1000). If the State granted double 
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credit for placing long-term welfare recipients, the djusted entered 
employment rate would be 72 prcent ([120 + 600]/1000). The SDA would have 
to be sure to maintain its unadjusted data for reporting on the JASR if such 
a system is used, but it could incorporate such a system for making 
incentive awards. An important feature of the extra credit approach is that 
it provides the SDAs with strong incentives to make sure that the designated 
participants are placed. The disadvantage is that States and SDAs would 
have to maintain two sets of outcome records-- actual accomplishments and 
adjusted data for incentive award calculations. 

Appendix 2 shows how an extra credit system can be developed that 
provides equivalent adjustments to the regression adjustment models, but wa 
believe that such a system is too complex to be feasible. A more practical 
use of the extra credit concept is as a supplement to the adjustment models, 
as was described above. Under such a system, SDAs would have their 
performance increased for positive outcomes for members of the selected 
group. One advantage of such a system is that it encourages SDAs to serve 
the targeted groups and it gives them a higher performance score when the 
services are successful. Friedlander and Long (1987) note that an extra 
credit system can also be used as a means of relating performance standards 
to net impact. 

4. Modifying factor weights for hard-to-serve groups . The weights in 
the Department of Labor's regression models are statistically derived 
regression coefficients. If the models are properly specified, these 
weights lead tc adjustments that hold SDAs harmless for serving groups 
included in the model. However, States may wish to change the weights for 



46 

two reasons. First, they may believe that the nationally derived weights 
are inappropriate for their State. Second, they may wish to go beyond 
holding SDAs harmless for serving particular groups and provide an 
incentive. Both rationales are considered below. 

Some of the weights that have emerged from the regression analyses 
have simply not conformed with common sense. The Department of Labor's TAG 
notes that the weights derived for heads of households and single parents 
did not have the expected sign, and these factors were dropped frcm the 
models. In other cases the estimated weights may have the right sign but 
still be incorrect. The weights may be incorrect because of s cification 
errors in the models (e.g., missing variables or measurement errors) or 
because of regional differences in the relationships (which i*; technically 
another type of missing variable problem) . States may seek to modify the 
weights of one or more factors to conform better with their /iew of what the 
weight should be. For example, Massachusetts reestimated the average wage 
at placement model and the adult welfare entered employment rate model to 
have tha same weight for all minority groups rather chan separate weights 
for blacks, Asians, and Hispanics because they believed that a single weight 
for all minority groups is more appropriate for their SDAs. States that 
believe that the model-derived weights are inaceir ate for their particular 
State should consider some of these alternatives. However, it should be 
kept in mind that if States make changes in the weights, their system is 
subject to Department c v Labor scrutiny if challenged. 

Changes in the weights might also be considered for policy purposes. 
As noted above, a State may wish to go beyond holding SDAs harmless for 
serving selected groups. Modifying the weights is one method to grant such 
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incentives, and it is likely to be more effective than reducing the standard 
through a Governor 1 s adjustment. For example, Massachusetts changed the 
length of stay weight in the cost model to encourage the use of intensive or 
long-term services. As was noted earlier, lowering the standard is likely 
to have a significant impact on SDAs in danger of being sanctioned, but 
changing the weight encourages services to the group of interest for all 
SDAs. 

The main arguments against permitting policy-based adjustments to the 
model weights are that the weights would no longer be objectively 
determined, and permitting variations for policy purposes could lead to 
abuse in some instances. By using the regression modeling approach, the 
Department of Labor has attempted to maintain neutrality on who is served in 
the programs. On the other hand, provisions in the statute encourage 
services for those most in need and permit Governors to provide monetary 
incentives for serving hard- to-ser\ groups. It can be argued that 
permitting Governors to adjust the weights is Oi:e such method of encouraging 
services to the hard- to -serve. 

5 . Providing technical assistance for encouraging services to the 
hard-to-serve . One use of a State's six-percent funds is to provide 
technical assistance to SDAs, including preventat e technical assistance 
for program improvement. Technical assistance can be used to explain which 
individuals the State considers hard-to-serve and the policies the State has 
implemented to encourage the provision of services to such individuals. 
Whenever a State adopts nev priorities or modifies its performance standards 
system substantially, technical assistance is important. If a State makes 
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use of a procedure that has not been widely used previously or is not 
described in the TAG, technical assistance is especially important. 
V- Summary and Conclusions 

Like many Federal programs, JTPA has many goals, and they are 
sometimes in conflict. We have considered how interested parties at the 
Federal, State, and SDA levels can deal with the somewhat competing goals of 
achieving high performance levels and serving the hard- to -serve. The 
statute offers a great deal of flexibility to all levels in the system, and 
we believe that promoting high performance is -ot inconsistent with 
promoting services to the hard- to- serve. 

All levels of JTPA play a role in determining who the system serves. 
The Federal government has developed adjustment models that hold SDAs 
harmless for serving individuals with certain characteristics. States play 
a major role by adding additional standards, adopting particular performance 
standards systems, and granting inc.i xive awards for serving particular' 
hard-to-serve groups. Ultimately, it is the SDAs, the PICs, and the service 
providers that respond to these incentives and their own policies and 
determine who is actually served in the program. Thus, all levels play 
important roles in determining the extent that the hard-to-serve are 
provided training opportunities in JTPA. 

With so many interested parties, it is not surprising that there is 
not unanimity in defining who the hard-to-serve are. Thus far, JTPA has 
made use primarily of demographic characteristics to adjust expected 
performance. There has been increasing interest in defining hard- to -serve 
status in terms of aeficl^eies and barriers. The Department of Labor has 
been seriously exploring methods of incorporating deficiency factors into 
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the adjustment models, particularly for basic skills and minimal work 
history deficiencies, as well as for long-term welfare recipients. Some 
States have also sought to encourage services to individuals with 
deficiencies by including adjustments for activities in their models. The 
great diversity in approaches among the States is encouraging, and points to 
the importance of defining hard-to-serve status. 

We have also identified a number of methods that can be used to 
encourage the provision of services to the hard- to- serve. Some choices 
cannot be avoided; for example, States must decide whether to use the 
national standards or the optionr L adjustment models. Other methods require 
more developmental work. Additional standards can be developed to encourage 
services to certain types of individuals, and performance standards can be 
integrated with level-of -service standards. We have also suggested some 
more innovative methods for consideration. The use of policy adjustments 
for factor weights in the models and extra credit for hard-to-serve 
individuals offer new optioi.^ We hope that this report has stimulated 
interested policy officials at all levels to consider and discuss the 
methods proposed here. 

The themes presented above are quite straightforward. First, the 
actors in the JTPA system need to take stock of which groups in the eligible 
population they wish to give priority. We have distinguished the hard-to- 
serve from broader groups that are dif ficult-to-place. Policy officials at 
the Federal, State, and SDA levels need to continue making progress toward 
defining the hard-to-serve in terms of key deficiencies such as a lack of 
basic skills and minimal work experience. Second, decisions must be made on 
how to structure the performance standards system so that at a minimum it 
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holds SDAs harmless for serving such individuals. 

We have presented a number of options for consideration: adding new 
factors for hard-to-serve status to the. model, modifying weights in the 
models, providing exnra credit for designated groups, and exempting 
innovative projects for the hard-to-serve from the standards are a few of 
the methods suggested. By considering these innovative practices, the JTPA 
system can continue to stress the importance of performance but also take 
steps to accommodate the extra resources required for serving clients who 
are hard-to-serve. 
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Notes 



1. Sandell and Rupp (1987) state that creaming may be defined as 
discrimination in program enrollment against individuals with 
particular characteristics that characterize them as being most in 
need. They then note that the most common criteria used for defining 
most in need are (1) demographic characteristics such as race and sex, 
(2) preprogram family income, and (3) employability of the applicants. 

2. The issue of making tradeoffs between the goals of equality and 
efficiency is discussed in detail by Okun (1968). Okun argues that 
while efficiency is important, society is also concerned with 
equality, and that trading off some economic efficiency to achieve 
better equity is often worthwhile, so long as the tradeoff is recognized. 

3. Procedures for adjusting performance standards must be: responsive to 
the intent of the Act; consistently applied among SDAs; objective and 
equitable throughout the State; and in conformance with widely 
accepted statistical criteria. Source data must be of public use 
quality and available upon request. Results must be documented 
clearly and reproducible. Adjustment factors must be limited to: 
economic factors, labor market conditions, characteristics of the 
population to be served, geographic factors, and types of services to 
be provided. 

4. The live criteria for being economically disadvantaged as specified in 
Section 4 of JTPA are: (A) receives, or is a member of a family which 
receives, cash welfare payments under a Federal, State, or local 
welfare program; (B) has, or is a member of a family which has, 
received a total family income for the six-month period prior to 
application for the program involved (exclusive of unemployment 
compensation, child support payments, and welfare payments) which, in 
relation to family size, was not in excess of the higher of (i) the 
poverty level determined in accordance with criteria established by 
the Director of the Office of Management and Budget, or (ii) 70 
percent of the lower living standard income level; (C) is receiving 
food stamps pursuant to the Food Stamp Act of 1977; (D) is a foster 
child on behalf of whom State or local government payments are made; 
or (E) in cases permitted by regulations of the Secretary, is an adult 
handicapped individual whose own income meets the requirements of 
clause (A) or (B), but who is a member of a family whose income does 
not meet such requirements. 

5. Examples of individuals who have barriers to employment are provided 
in Section 203 of the statute: individuals with limited English- 
language proficiency, displaced homemakers, school dropouts, teenaged 
parents, handicapped individuals, older workers, veterans, offenders, 
alcoholics, and addicts. 
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6. Sandell and Rupp point out that most of the people who are technically 
eligible for J1PA are very unlikely to be interested in the program. 
For example, of the 39 million people eligible for the program, about 
7 million are under age 16 or over age 64 and are unlikely to be 
interested in the program . Of the 31.7 mill ion eligible people 
between the ages of 16 and 64, Rupp and Sandell estimate that 13.0 
million are employed and another 14.8 million are not in the labor 
force. Over two -thirds of the JTPA participants are drawn from the 
3.9 million eligible individuals who are unemployed. 

7. The requirement that 40 percent of the funds be spent on youth is a 
national figure. For each SDA the percentage is to be adjusted 

depending on the ratio of disadvantaged youth to adults in the SDA. ♦ 

8. For elaboration on this point see Curnan and Fiala (1986). 

9. The JTPA experiment being funded by the Employment and Training 
Administration will be carried out in approximately 15 to 20 sites. 
Because of the small number of sites and the fact that the sites were 
not selected randomly, the findings may not be representative of the 
JTPA system as a whole. 

10 . A related technical issue is that when factors are measured with 
error, the estimated weights are often biased. See Kmenta (1971) for 

a discussion of the effects of measurement errors in regression analysis. 

11. Interaction terms are used if the effects of two characteristics are 
not additive. To illustrate the concept, suppose the model included a 
factor for race (white and nonwhite) and a factor for sex (male aiid 
female). If these were the only factors in the model, the regression 
equation would include two factors and would be of the following form: 

Y = £ N N + ftpF, where N is the percent nonwhite and F is the 
percent female. It is possible that the weights should be different 
for each group. In that case, the equation would include a third 
variable for one race-sex group, for example, nonwhite females. The 
model would then be: 

Y « £ N N + £pF + %pNF, where NF is the percent nonwhite 
females. 

12. Even if all deficiencies and barriers could be identified and included 
in the regression adjustment models, one v/ould still expe _ to find 
negative weights because of labor market discrimination faced by 
members of Aese groups . 

13. Other arguments can also be made for including activity variables in 
the models. Trott et al (1987) note that the selection of activities 
is not purely a management decision in the short run; there are a 
limited number of service providers that SDAs can use in the short 
run, so it might be unfair to penalize them for using the only 
providers available. They also note that by running regression models 
without variables that affect the performance measures, the other 
weights are biased. 
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It must be recognized that regional modeling can also lead to smaller 
weights for target groups. For example, the Illinois model for the 
adult entered employment rate has a weight of -.000609 for females, 
compared to a weight of -.02 in the national model. 
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Appendix 1 

Illinois Pilot Performance Standards Adjustment Models 




Region V Performance Standards Worksheet 



A. Service Oellvery Area's 
Name 



B. SOA Number 



C. Performance Period 



Type of Standard 

[ ] Plan 

f 1 Recalculated 



DATE 

Calculated 



E. Performance Measure 

Cost Per Positive Termination (Youth) 



F. LOCAL FACTORS 


G. SDA 
' FACTOR 
VALUES 


H. REGIONAL 
AVERAGES 


I. DIFFERENCE 
(6 HINUS H) 


J. WEIGHTS 


K. EFFECT OF LOCAL FACTORS 
ON PERFORMANCE EXPECTA- 
TIONS (I TIMES J) 


High School Grad or GEO 




29.82 




3.1873 




Act 1 - CRT/OCC Part 




19.17 




11.5757 




Act 6 - Youth Serv Part 




24.76 




-2.8407 




Lenqth of Prog Stay (Weeki) 




21.95 




11.1089 




Awage 85 - Avg Wage In Area PY'85 




18.29 




90.4199 










































L. fOTAL 






M. ILLINOIS DEPARTURE POINT 


$2,450.70 




N. Model -Adjusted Performance 
Level (L ♦ H) 





0. Performance Tolerance Range 

Bands ± 1300 



Performance Tolerance Range: Lower Band 



Planned/Actual 
Performance Outcome 



Upper Band 



nr 

4 \ t 1 
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Region V Performance Standards Worksheet 



C Performance Period 



Type of Standard 

[ ] Plan 

f 1 Recalculated 



OATE 

Calculated 



A. Service Delivery Area's 
Name 



B. SOA Number 



E. Performance Measure 

Positive Termination Rate (Youth) 



F. LOCAL FACTORS 


6 . SDA 
FACTOR 
VALUES 




f nTcccorurr 
I . UlrrtKtNlt 

id iiiuiic u\ 
(o mrlUa n) 


"1 LIT I^IITT 

J. WEIGHTS 


K. EFFECT OF LOCAL FACTORS 
ON PERFORMANCE EXPECTA- 
TIONS (I TIMES J) 














'Black 




49.11 




-.02783014 




Post HS 




6 67 




. U J Jo <:4 Jb 




Dropout 








Udo;i9H44 




Welfare 




37.54 




-.07326586 




Teen Parent 




6.31 




-.01333108 




Rural 




20.66 




-.05457839 




Inter 40 - Unemo 18-21 




36.88 




-.007192117 




Act 1 - CRT/OCC Part 




21 .14 




-.04654758 




Act 2 - CRT/Other Part 




10.53 




-.005091176 




Act & - Ex Youth Part 




34.25 




L .06591376 




Long 26 - Rec CRT > 26 weeks 




7.40 




-.01259350 




"MR - Employment Growth Rate 




9.44 




.44431175 









































TOTAL 



ILLINOIS DEPARTURE POINT 



0. Performance Tolerance Range 
Bands 



N. Model -Adjusted Performance 
Level (L f H) 



± 2.23% pts. Performance Tolerance Range: Lower Band 



jo - 



Planned/Actual 
Performance Outcome 



72.92 



Upper Band 



Region V Performance Standards, Worksheet 



A. Service Delivery Area's 
Name 



B. SOA Number 



C. Performance Period 



D. Type of Standard 
[ ] Plan 
f 1 Recalculated 



DATE 

Calculated 



E. Performance Measure 

Entered Employment Rate {Youth) 



F. LOCAL FACTORS 


6. SOA 
' FACTOR 
VALUES 


H. ILLINOIS 
AVERAGES 


I. DIFFERENCE 
(6 HINUS H) 


J. WEIGHTS 


K. EFFECT OF LOCAL FACTORS 
ON PERFORMANCE EXPECTA- 
TIONS (I TIMES J J 


Aqe 14-15 




8.35 




-.29476627 




Aqe 18-21 




61.37 




.04494737 




Black 




49.11 




-.04620763 




Student 




41.61 




-.05327057 




Dropout 




24.07 




-.12755792 




Welfare 




37.54 




-.06932456 




Rural - Hon Metro Residence 




20.66 




-.14247826 




"een Parent 




6.31 




-.06452865 




Handicapped 




14.29 




-.15K j198 




Act 1 - CRT/OCC Part 




21.14 




-.06283858 




Act 2 - CRT/Other Part 




10.53 




-.18473391 




Act 4 - Work Exp 




6.14 




-.31200862 




Act 5 - Emplv Services 




44.03 




.02172690 




t:l 6 - Ex Youth Part 




34.25 




-.22822208 




Lonq 26 - Rec CRT > 26 weeks 




7.40 




.09922325- 




EMGR - Employment 6rowth Rate 




9.44 




.16645507 






















































L. TOTAL 






H. ILLINOIS DEPARTURE POINT 


28.52 




N. Model -Adjusted Perfo. <ance 
Level (L ♦ Ml 




0. Performance Tolerance Range 

Rands + 2.762 pts. Performance Tolerance Range: Lower Band Upper Band 



P. Planned/Actual 

Perfonnance Outcome 
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Region V Performance Standards Worksheet 



A. Service Delivery Area's 
Bame 



B. SOA Number 



C. Performance Period 



0. Type of Standard 
[ ] Plan 
f 1 Recalculated 



DATE 

Calculated 



E. Performance Measure 

Cost Per Entered Employment (Adult) 



F. LOCAL FACTORS 


6.. SOA 
FACTOR 
VALUES 


H. REGIONAL 
AVERAGES 


I. DIFFERENCE 
(6 HINUS H) 


J. HEIGHTS 


K. EFFECT OF LOCAL FACTORS 
ON PERFORMANCE EXPECTA- 
TIONS (I TIMES 3) 


Black 




18.99 




15.7253 




Dropout 




23.91 




25.4670 




'Exhausted UC 




10.64 




32.3339 




Act 1 - CRT/OCC Part 




31.65 




24.1573 




Inter 12 - Hale Welfare 




16.03 




15.1977 




Unerep 85 - Ctv Unewo Rate PY'85 




9.3 




37.5387 




Awaqe 85 - Mvg Area l^age PY'85 




18.29 




77.4634 




Employment Growth Rate 




8.04 




2.1515 










































L. TCTAL 






M. ILLINOIS DEPARTURE POINT 


$3,723.65 




H. Model -Adjusted Performance 
Level (UN) 





Bands 



± *4S5 



Performance Tolerance Range: Lower Band 



P. Planned/Actual 

Performance Outcome 
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Region V Performance Standards Worksheet 


A. Service Delivery Area's B\ SDA Number 
Name 


C. Performance Period 0. Type of Standard 

[ ] Plan 
r 1 Recalculated 


OATE 

Calculated 


E. Performance Measure 

Average Wage at Placement (Placement) 




F. LOCAL FACTORS 


6. SDA 
' FACTOR 
VALUES 


H. ILLINOIS 
AVERAGES 


I. DIFFERENCE 
(G MINUS H) 


J. WEIGHTS 


K. EFFECT OF LOCAL FACTORS 
ON PERFORMANCE EXPECTA- 
TIONS (I TIMES «?) 


Female 




48.36 




-.008611286 




Inter 13 - Black Male 




15.00 




-.006454484 




1 ispanic 




12.74 




-.002532797 




1 andi capped 




4. 84 




-.004449971 




Dropout 




23.07 




-.003292346 




Post HS 




25.07 




.004268179 




Welfare 




32.84 




-.000886104 




UC None 




80.16 




- 006006708 




Act 1 - CRT/OCC Part 




35.32 




.v!02386025 




Act 2 - CRT/Other Part 




3.72 




-.001040204 




Act 3 - OJT 




35.64 




.003689562 




Lona 26 - Rec CRT > 26 weeks 




6.63 




.003508746 




Inc Pov 




7.40 




-.04762767 




Ann Earnings in Whs! 3st Trade 




12.11 




.02200934 




Pop Density 




.86 




.15167458 




































1 






L. TOTAL 






M. ILLINOIS DEPARTURE POINT 


4.72 




N. Mcdel -Adjusted Performance 
Level (Ud) 




0. Performance Tolerance Range 

RaruK ♦ $.136 Performance Tolerance Range: Lower Band Upper Band 



P. Planned/Actual 

Performance Outcome 



Region V Performance Standards Worksheet 



C. Performance Period 



0. Type of Standard 
[ ] Plan 
f 1 Recalculated 



DATE 

Calculated 



A. Service Delivery Area's 
Name 



D. SDA Number 



E. Performance Measure 

Welfare Entered Employment Rate 



' . LOCAL FACTORS 



Age 22-29 



Female 



Black 



6. SOA 
* FACTOR 
VALUES 



H. ILLINOIS 
AVERAGES 



51.96 



61.04 



51.74 

3.92 



I. DIFFERENCE 
(6 MINUS K) 



J. WEIGHTS 



.03231054 



.001332934 



-.09399677 



K. EFFECT OF LOCAL FACTORS 
ON PERFORMANCE EXPECIA- 
TIONS (J TIMES J) 



Dropout 



28.20 



-.06340624 



-.06495992 



Displaced H omenta leer 



Rural 



5.25 



24.11 



-.01367768 



-.10826685 



Act 1 - CRT/OCC Part 



Act 2 - CRT/Other 
Act 3 - OJT 



49.82 



5.00 



-.01308099 



-.14093117 



Long 26 - Rec CRT > 26 weeks 



21.21 



Employment Growth Rate 



10.09 



.20275285 



9.44 



-.08185879 



■11267559 



TOTAL 



0 Performance tolerance Range 
Bands 



M. ILLINOIS DEPARTURE POINT 



N. Model -Adjusted Performance 
Level fL » M) 



± 4.13* pts. Performance Tolerance Range': Lower Band 



P. Planned/Actual 

Performance Outcome 



50.7 



Upper Band 



Region V Performance Standards Worksheet 



A. Service Delivery Area's 
Name 



B. SOA Number 



C. Performance Period 



Type of Standard 

[ ] Plan 

f 1 Recalculated 



DATE 

Calculated 



E. Performance Measure 

Entered Employment Rate (Adult) 



P 1 flPAl CAPTHD^ 


• FACTOR 
VAI UFS 


H III INOIS 

• It ILLlnUl J 

*VFBAGFS 


I DIFFERENCE 
(G MINI'S Hi 


J WEIGHTS 


K EFFECT OF LOCAL FACTORS 
ON PERFORMANCE EXPECTA- 
TI0NS (I TIMES J) 


PprfiA 1 p 
r ctiMi i c 




50.0 




- .000609424 




Black 




35.36 




-.09024531 




Ha nH \ c 3k nnpH 




5 67 




-.12961032 




Dropout 




24.92 




-.05704486 




Post HS 




24.20 




.01135771 




Welfare Recipient 




38.96 




-.12043240 




Offender 




6.53 




-.02269636 




Act 1 - CRT/OCC Part 




39.69 




-.08778202 




Act 2 - CRT/Other Part 




4.68 




-.V 391 5331 




Long 26 - Rec CRT > 26 weeks 




u.41 




-.09619200 




Employment Growth Rate 




9.4* 




.46693783 




Labor Force Part Rate 22-44 




82.07 




.20613981 










































L. TOTAL 






H. ILLINOIS DEPARTURE POINT 


61.11 




N. Node 1 -Adjusted Performance 
Level (L * H) 





0. Performance Tolerance Range 
Bands 



f 2.37X pts. Performance Tolerance Range: Lower Band 



Upper Band 



P. Planned/Actual 

Performance Outcome 
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Appendix 2 

Extra Credit for Participants tfith Barriers to Employment 
I- Introduction 

The primary method currently us^d to encourage services to hard-to- 
serve clients in JTPA is the Department of Labor ! s regression adjustment 
models. The regression models are used to adjust the levels of minimum 
required performance on the seven measures established by the Secretary. 
There are several alternative ways that incentives can be provided to SDAs for 
serving particular client groups. In this append: ; we discuss the use of 
"extra credit" in computing performance as a means of encouraging services to 
hard-to-serve groups. 

The term e: **ra credit can be used in several ways in a performance 
standards context, but we will use the term to describe a procedure where SDAs 
are given a higher score on one or more performance measures for terminees 
with particular characteristics. For example, in computing performance on the 
entered employment rate, an SDA could be given credit for 1.5 or 2.0 people 
entering employment for each member of a particular hard-to-serve group that 
enters employment rather than receiving credit for one person. Similar 
adjustments can be made for other outcomes of interest; for example, wages for 
those entering employment can be increased. We are not considering here the 
provision of extra six-percent funds for serving members of particular groups. 

The extra credit approach can be used Instead of the regression 
adjustments for v^iables already in the models, for other factors not 
included in the adjustment models, or in addition to the adjustments made by 
the models. After discussing the advantages and disadvantages of an extra 



credit system, we show how the mechanics of using such a system can operate. 

II- Advantages and Disadvantages of an Extra Credit System 

The extra credit system offers several potential advantages over the 
regression adjustment model currently used. First, the extra credit system 
enables SDAs to raise their performance scores when serving hard- to -serve 
individuals rather than lower the minimum acceptable performance. Wo have 
heard that some individuals in the employment and training co.jmuni.ty are more 
concerned about the level of outcomes rather than the level adjusted for the 
mix of participants, although it is not clear how widespread this phenomenon 
is. Thus, an SDA might prefer to have its performance adjusted higher than to 
have its standard lowered because it is serving hard-to-serve clients. 

A second advantage of the extra credit approach is that it provides more 
year-to-year continuity for judging performance and a common basis for 
comparing SDAs within a State. Under the regression podel adjustment system, 
one cannot judge whether an SDA has improved its performance by looking at its 
scores in consecutive years because the mix of participants may hav*. changed. 
Under an extra credit system, assuming that the credit has been awarded 
appropriately, the performance each year can be compared with performance in 
previous years. In addition, SDA performance scores can be compared with one 
another directly; with the regression adjustment model system, SDAs can only 
be compared by looking at the extent to which each SDA exceeded its standards. 

Another advantage of the extra credit system is that the relative 
incentives for serving various types of individuals are likelier to be 
understood under an extra credit system than under the regression model 
adjustment system. For example, the incentives provided by the weights in the 



A-3 

model arc more dif f icu) * to interpret than getting extra placement credit. 

A final advantage of the extra credit method is that it is likely to 
provide a stronger incentive for SDAs to find jobs for members of the target 
groups of interest. Under the regression model adjustment system, an SDA has 
an incentive to enroll individuals in groups with negative weights, but once 
they are enrolled the SDA gets no more credit for placing such individuals 
than it does for placing others. Xn an extra credit system, the SDA has an 
* incentive to give extra effort to finding a person in the extra credit 

category a job since the SDA gets more credit for placing the hard-to-serve 
individual . 

The extra credit system also has several disadvantages that should be 
kept in mir\d. If a State already uses the regression model approach, then 
using the extra credit approach requires additional computations by the State 
to remove the factors from the model and provide extra credit instead. Also, 
using a combination of regression adjustments and extra credit adjustments at 
the same time is likely to confuse State and SDA officials and make 
interpretation of SDAs 1 scores difficult. 

Ill- Developing an Extra Credit Equivalent for Factors in the Model 

In chis section we show how a factor that is currently included in the 
regression model can be accounted for by giving extra credit instead. The 
general procedure can be used for any variable in the model, and we have 
applied it to percent of term.\nees on welfare. Extra credit can also be 
granted for factors not included in the regression model, but an alternative 
procedure must be used to determine how much extra credit to grant. 

To illustrate how to switch from the regression model adjt >tment to an 

ERIC SO 
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equivalent standard using extra credit, we make use of the sample worksheet 
for the adult entered employment rate standard reproduced in Exhibit 1 of the 
text. From Line 8 of the worksheet, the weight for percen** welfare recipient 
is -.276. In the worksheet, the hypothetical SDA had a model -adjusted 
performance standard of 60,1 percent for its adult entered rmployment rate. 
If the SDA had 100 percent welfare recipients and no other factors changed, 
its model- adjusted performance level would be 44.1 percent. (This can be seen 
by substituting 100 for 42 in row 8 column G and carrying through the « 
arithmetic.) If the SDA had no welfare recipients, its ^odex- adjusted 
performance level would be 71.7 percent, again assuming no changes in any 
other factors. The adult entered employment rate standard for the SDA can be 
viewed as a combination of 44.1 percent for welfare recipients and 71.7 
percent for nonwelfare terminees; wi-h 42 percent of the terminees in welfare, 
the standard can be calculated as: 

(.42 x 44.1) + (.58 x 77.1) - 60.1 percent, the same figure as shown on the 
sample worksheet. 

To reolace the model -adjustment approach with exira credit for serving 
welfare recipients, would proceed in several steps. Since an SDA would gee 
extra credit for placing welfare recipients, the standard to be met should not 
depend on the number of welfare recipients served. Thus, the entered 
employment rate standard should be set at 71.7 percent, the level the SDA 

y 

would have to achieve if it served no welfare clients. 

The next step in the process is to determine how much extra credit to " 
give for placing welfare clients. The extra credit should be enough so that 
an SDA that exactly met the standard before would meet the new standard with 
the extra credit. For each welfare person v,no enters employment tfte SDA 
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should get credit for the standard for nonwelfare clients, 71.7 percent in 
this example, divided by the model -derived placement rate for welfare clients, 
44.1 percent in this case. Thus, for each welfare client who enters 
employment, the SDA should get credit for 71.7/44.1 - 1.63 individuals who 
enter employment. 

To illustrate how the extra credit calculations work, we will consider 
several examples. As a simple case, consider an SPA that served no welfare 
,* recipients before. Under the regression-adjustment system its standard would 

be 71.7 percent, and it would have the same standard using extra credit to 
reward serving welfare clients. 

Next consider the SDA from the sample worksheet. This SDA faced a 
model -adjusted performance level of 60.1 percent. Assuming it exactly net its 
standard by having an entered employment rate of 44.1 percent for welfare 
recipients and 71.7 percent for nonwelfare clients, we can see how the SDA 
fares under the extra credit system. Under the extra creuit system the 
placement rate for nonwelfare terminees remains the same, but the rate for 
welfare terminees is multiplied by 1.63. The adjusted entered employment rate 
for the SDA is now 71.7, the seme as Its standard. 

An interesting feature of the extra credit approach is that an SDA's 
performance is sensitive to the composition of those placed. In the example 
just noted, if the SDA had the same overall entered employment rate but the 
rates for welfare and nonwelfare terminees are varied, the adjusted entered 
* employment rate of -he SDA would be affected. In particular, if the rate for 

welfare terminees ncreased while the rate for nonwelfare terminees decreased, 
the SDA would receive e higher score than if the situation were reversed. 
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IV- Summary and Conclusions 

Offering extra credit for performance associated with hard-to-serve 
groups is a viable method of encouraging the provision of services to 
individuals with barriers to employment. As we have seen, an extra credit 
adjustment system can replace the regression model adjustment system for one 
or more adjustment factors. Using extra credit for some factors and 
regression adjustments for others might be confusing for some States and SDAs. 
One very appealing feature of the extra credit approach is that it directly 
enhances an SDA's performance rather than reducing what is expected of the 
SDA. Thus, it is a more positive approach than the regression adjustment 
model . 

The examples above all deal with how the axtia credit approach can be 
used to compensate for expected poorer performance for hard-to-serve groups. 
Extra credit can also be used to add incentives for SDAs to serve members in 
certain groups. For example, a State could use the regression model to 
account for the difficulty in placing welfare clients, but it could also 
provide some extra credit as an added inducement to serve and piace welfare 
events. Also, for factors that have not formally been modeled, extra credit 
can be used in l:eu of or in addition to a subjective weight to encourage 
serving particular groups. While lacking the mathematical properties of a 
model-derived adjustment, such adjustments might help States accomplish their 
objectives . 



